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(57) The present invention provides a mobile phone 
that includes a single camera for picking up a 2D image 
of a subject, a 3D image creation section for providing 
the 2D image with parallax information to create a 3D 
image and a display unit for displaying the 3D image. In 
accordance with another aspect of the present inven- 
tion, a 3D conversion program is provided, comprising 
the steps of inputting a 2D image, cutting a human face 
image out of the 2D image to obtain a face image, pro- 
viding the face image with parallax information to create 
a 3D image and outputting the 3D image. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001 ] The present invention relates generally to mo- 
bile equipment and three dimensional (3D) conversion 
program products and particularly to mobile equipment 
and 3D conversion program products capable of readily 
providing a 3D display. 

Description of the Background Art 

[0002] In recent years, mobile equipment is increas- 
ingly miniaturized and enhanced in function. Further- 
more, mobile terminals, mobile personal computers, 
mobile phones and other similar mobile equipment are 
provided with a function of a camera. For example, cam- 
era-equipped mobile phones have recently started to 
prevail. They are convenient as they can for example 
set a picked-up image on an idle screen and transmit a 
picked-up image on-site to the called. 
[0003] Japanese Patent Laying-Open No. 1 0-1 081 52 
discloses a mobile information terminal which as part of 
enhancing mobile equipment in function utilizes parallax 
from both eyes to provide a stereoscopic display (a 3D 
display). Utilizing parallax from both eyes to provide a 
3D display can add depth to a stereoscopic image dis- 
played, providing a more realistic and lively image than 
a two dimensional image. As such, this technique is ap- 
plied in the field of videophone. The mobile information 
terminal disclosed in Japanese Patent Laying-Open No. 
1 0-1 081 52 is a mobile information terminal of a type pro- 
vided with two cameras and worn by a user on the arm. 
The mobile information terminal produces two video im- 
ages provided with a temporal difference depending on 
a horizontal component of a subject's motion picked up 
through the two cameras and provides the images to the 
viewer's right and left eyes to implement a 3D display. 
Note that "a 3D display" means providing a stereoscopic 
display utilizing parallax. 

[0004] The conventional mobile information terminal 
disclosed in Japanese Patent Laying-Open No. 
1 0-1 081 52, however, is not economical as the two cam- 
eras result in a higher price and an increased power con- 
sumption of the terminal. 

[0005] Desirably, the two cameras picking up an im- 
age which is in turn three dimensionally displayed 
through parallax of human both eyes should be spaced 
by a distance corresponding to the parallax and in that 
arrangement pick up an image of a subject. If the dis- 
tance between the two cameras is maintained while 
components otherthan the cameras are arranged, how- 
ever, the arrangement of the other components is limited 
by that of the two cameras and the mobile information 
terminal's miniaturization and degree of freedom in de- 
sign are impaired. 



[0006] Furthermore, a moving subject that is picked 
up through the camera and displayed in a 3D image 
would in effect hardly provide a sufficient image. In ad- 
dition, for a user normally using the camera to pick up 
5 a 2D image and, as required, pick up a 3D image and 
displaying the 3D image, only one of the two cameras 
would normally be used. 

[0007] In other words, providing two cameras to pro- 
vide a 3D display is accompanied by the aforemen- 
10 tioned problems of higher price, increased power con- 
sumption, and impaired miniaturization and design, and 
it has been an obstacle to practical utilization of a 3D 
function in mobile equipment terminals. For mobile 
phones, in particular, there is a demand for reduction in 
is size, weight, cost and the like, and it is difficult to provide 
mobile phones with two cameras. 

SUMMARY OF THE INVENTION 

[0008] Accordingly one object of the present invention 
is to provide mobile equipment that is reduced in size 
and power consumption and has a high degree of free- 
dom in design. 

[0009] The above object of the present invention is 
achieved by mobile equipment including the following 
components: a pickup device picking up an image of a 
subject; a three dimensional image creation portion pro- 
viding the image with parallax information to create a 
three dimensional image; and a display unit displaying 
the three dimensional image. 

[0010] In accordance with present invention in anoth- 
er aspect the mobile equipment includes: a single pick- 
up device picking up an image of a subject; a three di- 
mensional image creation portion providing the image 
with parallax information to create a three dimensional 
image; and a display unit displaying the three dimen- 
sional image. 

[001 1 ] In accordance with present invention in still an- 
other aspect a three dimensional conversion program 
product causes a computer to execute a three dimen- 
sional conversion process. It causes the computerto ex- 
ecute the steps of: inputting a two dimensional image; 
cutting a human face image out of the two dimensional 
image to obtain a face image; providing the face image 
with parallax information to create a three dimensional 
image; and outputting the three dimensional image. 
[001 2] The foregoing and other objects, features, as- 
pects and advantages of the present invention will be- 
come more apparent from the following detailed de- 
scription of the present invention when taken in conjunc- 
tion with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0013] In the drawings: 

Fig. 1 is a block diagram showing a main configu- 
ration of a mobile phone in an embodiment; 
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Fig. 2A is an overall view of a foldable mobile phone 
201 flipped open, as seen at its internal side (or a 
key bearing side), and Fig. 2B is an overall view 
thereof closed, as seen at its external (back) side; 
Fig. 3 is a plan cross section of a configuration of a 5 
main portion of a liquid crystal module capable of 
providing a 3D display that is used for a main screen 
203; 

Fig. 4 is a flow chart representing a process per- 
formed in the mobile phone to pick up and three di- io 
mensionally display an image of the face,of a per- 
son; 

Figs. 5A and 5B show a specific example of a 
screen displayed in accordance with a first embod- 
iment on main screen 203 two dimensionally and 15 
three dimensionally, respectively; 
Fig. 6 represents a concept for illustrating how to 
obtain parallax information in the first embodiment; 
Figs. 7A and 7B show a specific example of a 
screen displayed in accordance with a fourth em- 20 
bodiment on main screen 203 two dimensionally 
and three dimensionally, respectively; and 
Fig. 8 is a block diagram showing a configuration of 
a main portion of the mobile phone in a third em- 
bodiment. 25 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

First Embodiment 30 

[001 4] The present invention in a first embodiment will 
be described with reference to the drawings. While the 
present invention is applicable to any mobile equipment, 
hereinafter as a suitable embodiment a foldable, cam- 35 
era-equipped mobile phone will be described. 
[0015] Fig. 1 is a block diagram showing a main con- 
figuration of the mobile phone in the present embodi- 
ment. In the first embodiment the mobile equipment in- 
cludes: a communication control portion 101 communi- 40 
eating with a base station (not shown); an audio signal 
processing portion 103 receiving an audio signal from a 
microphone 102 and encoding the signal and also de- 
coding an audio signal received from a counterpart in 
the call of interest; a speaker 1 04 reproducing the de- 45 
coded audio signal; a processor 105 executing a pro- 
gram to generally control the mobile phone; a memory 

106 referenced by processor 105; a key operation unit 

107 receiving an input from a user; a camera module 

108 such as a CCD (charge coupled device) camera 50 
used for picking up a typical 2D image; a color liquid 
crystal module 1 09 for a main screen; and a color liquid 
crystal module 110 for a subscreen. 

[0016] Fig. 2A is an overall view of foldable mobile 
phone 201 flipped open, as seen at its inner side (or a 55 
key bearing side), and Fig. 2B is an overall view thereof 
closed, as seen at the outer (back) side. 
[0017] As shown in Fig. 2A, mobile phone 201 has 



speaker 104 arranged at the cover at an inner, upper 
center portion, and microphone 102 arranged at the 
body at an inner, lower center portion, which facilitate 
calls with mobile phone 201 open. Furthermore, opera- 
tion keys 204 including numeral keys, a cross key 209 
and the like are arranged at the body on an inner side 
and a main screen 203 is rectangular and arranged at 
the cover on the inner side so that with mobile phone 
201 open they provide superior operability. Further- 
more, a camera 206 is arranged at the cover on the outer 
side close to the hinge in the vicinity of the center and 
a subscreen 207 is rectangular and arranged at the cov- 
er on the outer side in the vicinity of the center so that 
even with mobile phone 201 closed the user can confirm 
a subject on subscreen 207 in picking up an image 
through camera 206. Furthermore, a back-side key 208 
is arranged under subscreen 207 so that it can be used 
as a shutter key with mobile phone 201 closed. While 
Fig. 2B shows only a single key 208, providing a plurality 
of keys 208 allows them to be used differently as keys 
operated to switch information displayed on subscreen 
207 (e.g., a scroll key, a key operated to select an image 
accumulated, and the like). If the user picks up an image 
of a subject with mobile phone 201 open, the user con- 
firms the subject on main screen 203 or subscreen 207, 
while the user uses any of operation keys 204 as a shut- 
ter key to pick up an image. 

[0018] Main screen 203 is configured of a liquid crys- 
tal module capable of a stereoscopic display (hereinaf- 
ter referred to as a 3D display). Alternatively, both or one 
of main screen 203 and subscreen 207 may be formed 
of a liquid crystal module capable of a 3D display. Al- 
lowing subscreen 207 to display a favorite 3D image as 
an idle screen enables the user to constantly see the 3D 
image without opening mobile phone 201 , which adds 
value to the mobile phone. 

[001 9] Fig. 3 is a plan cross section of a configuration 
of a main portion of a liquid crystal module capable of a 
3D display that is used for main screen 203. With refer- 
ence to Fig. 3, the liquid crystal module includes a back 
light 301 , a switching liquid crystal element 302 switch- 
able in deflection angle, a patterned phase difference 
plate 303 patterned in a slit, and a thin film transistor 
(TFT) liquid crystal 304. 

[0020] To display a 3D image on main screen 203, the 
liquid crystal element 302 deflection angle is switched 
to change a relationship of the liquid crystal element 302 
phase and the phase difference plate 303 phase to in- 
terrupt an image for the left eye to prevent the image 
from passing through an L pixel 306 (displayed by a 
white pixel in the figure) and being projected on the right 
eye (a broken line 309). Likewise, an image for the right 
eye is interrupted so that the image is prevented from 
passing through an R pixel 305 (displayed by a black 
pixel in the figure) and being projected on the left eye (a 
broken line 310). 

[0021] Thus on the viewer's right eye the image for 
the right eye alone is projected and on his/her left eye 
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the image for the left eye alone is projected. This pro- 
vides the viewer's right and left eyes with different im- 
ages and as a result the viewer can perceive a three 
dimensional, stereoscopic image (a 3 D image) provid- 
ing a sense of depth. 

[0022] Furthermore, if a main screen 203 displays a 
2D image (a normal, two dimensional planar image), the 
above-described optical interruption by liquid crystal 
302 and phase difference plate 303 is not provided. 
Electrically controlling liquid crystal 302 and phase dif- 
ference plate 303, as described above, allows main 
screen 203 to display a 2D image and a 3D image, as 
switched. 2D and 3D images can simply be switched for 
example by lightly pressing any operation key 204, such 
as a power supply key. 

[0023] Note that while in the present invention a liquid 
crystal module configured as described above is used 
to provide a 3D display on main screen 203, a configu- 
ration using a lenticular lens, a configuration using con- 
tinuous grain silicon liquid crystal rather than TFT liquid 
crystal 304 or a similar configuration may be used to do 
so. Furthermore, while the above described configura- 
tion employs for main screen 203 a liquid crystal module 
capable of a 3D display, a typical liquid crystal module 
allowing main screen 203 to display a 2D image may be 
used and the display may be observed through polari- 
zation glasses. In that case, a configuration combining 
together the liquid crystal module allowing a 2D image 
to be displayed and the polarization glasses provides a 
display means capable of a 3D display. 
[0024] Hereinafter will be described a procedure of a 
process using the thus-configured mobile phone to pick 
up a 2D image through camera 206, create a 3D image 
therefrom and finally display the 3D image on main 
screen 203. 

[0025] By way of example, an image of the face of a 
person is picked up and three dimensionally displayed, 
as will be described hereinafter with reference to the 
flow chart shown in Fig. 4, an example of a screen dis- 
played as shown in Figs. 5A and 5B, and the diagram 
shown in Fig. 6 representing a concept in obtaining par- 
allax information. 

[0026] With reference to Fig. 4, and Fig. 5A, a user 
confirms on main screen 203 the face of a person as a 
subject 501 , seen via a camera. When a desired image 
is displayed on main screen 203, the user presses op- 
eration key 204 that serves as a shutter key to pick up 
an image of subject 501 . Camera module 108 takes in 
the image in the form of a 2D image (a still picture) 

(5401) . 

[0027] Then by processor 1 05 a mode of whether fa- 
cial parts such as eyes, nose, mouth, eyebrows, jaw and 
the like and the face's contour are extracted from the 
taken 2D image automatically or manually is confirmed 

(5402) . This face image extraction mode may be set by 
the user previously for example via a menu screen of 
the mobile phone, or it may be set by the user whenever 
he/she operates the mobile phone. If the face image ex- 



traction mode is the manual extraction mode ("manual" 
at S402), processor 105 receives from the user via op- 
eration key 204 a designation of a position of each facial 
part as described above (S403). Then processor 105 

5 uses positional information designated at step S403 to 
obtain a two dimensional coordinate of each facial part 
(S404). In doing so, allowing a pen a mouse, a jog dial 
or the like (not shown) to be used for input to provide 
selectableness provides enhanced operability. Note 

to that when a face's 2D image is used to create a 3D im- 
age, the nose protrudes most stereoscopically. Accord- 
ingly, whether the face image extraction mode is the au- 
tomatic extraction mode or the manual extraction mode, 
the nose is set as a facial part to be always extracted. 

*s in particular, if human perception is considered, empha- 
sizing the protrusion of the nose can provide the face 
with a sense of depth. 

[0028] If the face image extraction mode is the auto- 
matic extraction mode ("automatic" at S402), processor 

20 105 employs a well known facial part extraction tech- 
nique for example utilizing edge detection to obtain a 
two dimensional coordinate of each facial part from the 
2D image taken atstepS401 (S404). For mobile phones 
or similar equipment inconvenient for the user to per- 

25 form an input operation, it is convenient that facial parts 
are automatically extracted. Furthermore, in picking up 
an image, displaying a round, square or similar frame 
on main screen 1 09 and allowing the user to pick up the 
image with a face image positioned in a vicinity of the 

30 frame allows processor 1 05 to extract facial parts in a 
narrowed range so that for the automatic extraction 
process can be performed rapidly with high precision. 
[0029] At step S404 each facial part's two dimension- 
al coordinate is obtained and when a face image is cut 

35 out of the 2D image taken in at step S401 processor 1 05 
then projects the extracted, each facial part's coordinate 
on a predetermined, three dimensional human face 
model to obtain three dimensional data of the entirety of 
the face image (S405). At step S405 the three dimen- 

40 sional model generation method previously filed by the 
applicant and disclosed in Japanese Patent Laying- 
Open No. 2001-109907 is used, as described hereinaf- 
ter. 

[0030] Processor 105 estimates (detects) the size, 
45 position and direction of the face from each facial part's 
two dimensional coordinate obtained at step S404 and 
projects a standard face geometry model, which is 
based on a standard human face geometry correspond- 
ing to a three dimensional, human face model, in the 
50 estimated size, position and direction on the 2D image 
taken in at step S401 . Processor 105 then calculates a 
positional offset between a coordinate of each facial part 
included in the standard face geometry model that is 
projected on the 2D image and each facial part's two 
55 dimensional coordinate obtained at step S404. Proces- 
sor 105 then projects the model in a position and direc- 
tion slightly offset from the provisionally estimated size, 
position and direction and similarly calculates a position- 
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al offset between a coordinate of each facial part includ- 
ed in the model that is projected and each facial part's 
two dimensional coordinate obtained at step S404. 
Processor 1 05 thus obtains a size, position and direction 
providing a minimal offset between the coordinates and 
uses it as a subsequent, provisional estimated size, po- 
sition and direction in repeating the above described 
process. Processor 105 repeats such a process and fi- 
nally when the offset is no longer reduced processor 1 05 
estimates the current size, position and direction as a 
final size, position and direction of a face image included 
in the 2D image. 

[0031] Processor 1 05 then projects on the 2D image 
taken at step S401 standard face geometry data in the 
thus-estimated size, position and direction to generate 
a texture image. More specifically, processor 1 05 initial- 
ly calculates coordinates of all apexes, respectively, of 
the standard face geometry data. Simultaneously, proc- 
essor 1 05 provides a parallel projection of the standard 
face geometry data on the texture image in a predeter- 
mined size and position at a front side and calculates 
coordinates of all apexes, respectively, of the standard 
face geometry data. More specifically, a coordinate of 
an apex projected on the 2D image is assumed to cor- 
respond to that of the same apex projected on the tex- 
ture image, and based on this correspondence an image 
conversion tech nique is employed to transfer each facial 
image from the 2D image onto the texture image. The 
image conversion technique used herein is a well known 
technique in the field of image processing and it can be 
readily implemented by those skilled in the art. As such, 
the image conversion technique will not be described in 
detail herein. 

[0032] The transfer provides an image approximate 
to an image of a subject included in the 2D image that 
is picked up at a front side to have a predetermined size 
and position. Processor 105 generates a three dimen- 
sional model corresponding to such an approximate im- 
age at step S405 as a texture image pasted on standard 
face geometry data at a front side by parallel projection. 
[0033] Note that at step S405, as has been described 
above, as a three dimensional, human face model a 
standard face geometry model based on a standard hu- 
man face geometry may be used. Alternatively, a plural- 
ity of face geometry models such as a round face, a thin 
face and the like may previously be prepared to allow a 
user to select a model. Furthermore, a dedicated face 
geometry model for a particular person can previously 
be stored in memory 1 06 and at step S401 an image of 
the person can be picked up and at step S405 the face 
geometry model dedicated to the person can be used 
to generate a more precise 3D image. For example, if a 
face geometry model of the owner of the mobile phone 
is stored in memory 106, the owner's more precise 3D 
face image data can be readily generated. The mobile 
phone shown in Figs. 2A and 2B allows the user to take 
his/her picture with the phone closed while on sub- 
screen 207 the user can confirm his/her face. The user 



can readily pick up his/her face image. If a plurality of 
people exchange their respectively produced 3D face 
images by means for example of attaching the images 
to mail, the high precision 3D face images can be en- 
5 joyed among more than one person. Note that a per- 
son's dedicated face geometry model is calculated 
through a large quantity of calculation and a result of 
calculation by a server may be downloaded to the mo- 
bile phone. 

io [0034] At step S405 a face image's three dimensional 
data is obtained. Processor 1 05 then provides the three 
dimensional data with parallax information (S406). At 
step S406 processor 1 05 performs an approximate cal- 
culation based on a standard distance between human 

*5 both eyes (e.g., approximately 6 cm) and a typical dis- 
tance selected when a mobile phone is used to take a 
picture of a human face (e.g., approximately 1 m), ob- 
tains an angle a formed by a line extending between the 
subject's face and one eye assumed at the position of 

20 the camera and a line extending between the subject's 
face and the other eye assumed at the position of the 
camera, and imparts to three dimensional data the par- 
allax information corresponding to angle a. More spe- 
cifically, as shown in Fig. 6, processor 105 provides as 

25 image data for the right eye the data obtained through 
projection from the right eye's provisionally assumed 
position onto a face image's three dimensional data and 
conversion and as image data for the left eye the data 
obtained through projection from the left eye's provision- 

30 ally assumed position onto the face image's three di- 
mensional data and conversion. Processor 105 thus 
creates a 3D image including an image for the right eye 
and that for the left eye (S407). 
[0035] Processor 1 05 then displays the 3D image on 

35 main screen 203. The viewer visually perceives a hu- 
man face projected out of main screen 203, as shown 
in Fig. 5B (S408). 

[0036] Note that although in Fig. 5B, because of draw- 
ing technique, the nose's height, the eyes' recess on 

40 other similar sense of depth are not represented, the 
viewer can perceive such depth and enjoy a more real- 
istic face image. Furthermore in Fig. 5B the person's 
neck, shoulders and therearound are also displayed 
similarly as shown in Fig. 5A. Thus the present mobile 

45 phone may be adapted to also cut out a portion of a per- 
son other than the face or put clothes or the like previ- 
ously prepared as a template on a person like a dress- 
up doll and display the same. 

[0037] While in the above example processor 1 05 cal- 
50 culates parallax information with a distance from cam- 
era 206 to subject 501 as a predetermined distance (for 
example of 1 m), the user may input a distance via op- 
eration key 204 whenever the user take a picture. If the 
user take a picture of his/herself, the user often reaches 
55 out his/her arm holding the mobile phone to take his/her 
picture. As such, the user taking his/her picture is closer 
(e.g., 50 cm) to the mobile phone than the face of a per- 
son other than the user. Accordingly, when the user 
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takes his/her picture, processor 105 may be adapted to 
automatically switch a predetermined distance in calcu- 
lating parallax information. Note that processor 105 can 
automatically determine whether a picture of the user's 
face or that of a person other than the user is to be taken 5 
for example by detecting whether the cover is open or 
closed. 

[0038] Furthermore while in the above example the 
face of a person is cut out of a picked-up 2D image, and 
displayed three dimensionally, a background image (e. 
g., a plane shown in Fig. 5A) excluding a human face 
may be displayed together with the face. A human face 
and a background image that are displayed together 
provide a further increased sense of depth and hence a 
more lively image. 

[0039] Furthermore while the above example has 
been described in conjunction with a three dimensional 
human face model, for example a Pomeranian or any 
other similar dog's three dimensional face model may 
be prepared to allow the user to create his/her pefs 3D 
image or a three dimensional human body model may 
be prepared and used to create a 3D image of a human 
entire body. 

[0040] In the above example after an image of a sub- 
ject is picked up main screen 203 displays a 2D image 
as shown in Fig. 5A. Alternatively, operation key 204 
may previously include a 3D shutter button and if the 
above described 3D conversion process is adapted to 
be automatically performed then after the 3D shutter 
button is pressed a 3D image as shown in Fig. 5B can 
immediately be displayed. 

Second Embodiment 

[0041] If a "PLAY3DPC" product kit available from l-O 
DEVICE, INC. is used, an image picked up through a 
digital camera can be taken into a personal computer 
(PC) and the image's each block can be analyzed in 
brightness by dedicated software. A brighter block is as- 
sumed to be positioned forward in creating a 3D image. 
The user then displays the created 3D image on a dis- 
play connected to the PC and via liquid crystal glasses 
views the 3D image. 

[0042] The technique utilizing an image's brightness 
to create a 3D image can be used to allow processor 
105 of the mobile phone in the second embodiment to 
divide a 2D image picked up through camera 206 into a 
plurality of blocks and detect each block's brightness, 
and provide parallax information for each block to allow 
a brighter block to have a more forward position to cre- 
ate and display a 3D image on main screen 203. For 
example, a perspective background image including an 
automobile positioned forward and a building backward 
can be displayed. 
[0043] In this case, processor 1 05 is driven by an in- 55 
tensity in brightness of each block of the 2D image to 
provide different parallax information for each block to 
create a 3D image. As such, an approximate calculation 



for calculating parallax information, as described in the 
first embodiment, is not required. 

Third Embodiment 

[0044] Japanese Patent Laying-Open No. 
2002-77944 discloses a technique, which can be used 
to illuminate a subject with light and detect from intensity 
of a reflection of the light a distance from the mobile 
phone to each portion of the subject, and be driven by 
the distance information to provide the subject's each 
portion with different parallax to create a 3D image. The 
created 3D image can be displayed on externally con- 
nected, stereoscopic video image display device. More 
specifically, in the mobile phone of the third embodiment 
the technique disclosed in Japanese Patent Laying- 
Open No. 2002-77944 can be adopted to allow proces- 
sor 1 05 to be driven by a distance to a subject of a 2D 
image picked up through camera 206 to provide each 
portion of the subject with different parallax information 
to create a 3D image and display the image on main 
screen 203. In this example the mobile phone in the third 
embodiment includes the components shown in Fig. 1 
plus a radiation unit 801 illuminating a subject with light 
and a detection portion 802 detecting the intensity of a 
reflection of the light, as shown in Fig. 8. 

Fourth Embodiment 

[0045] In a fourth embodiment the mobile phone may 
include processor 1 05 accepting in response to a user's 
selection via operation key 204 (e.g., cross key 209) a 
subject in a 2D image picked up through camera 206 
and displayed on main screen 203 that is located for- 
ward, and providing the selected subject with parallax 
information different than a background image other 
than the selected subject to create a 3D image. 
[0046] For example, as shown in Fig. 7A, the user us- 
es a selection frame 701 displayed on main screen 203 
to designate in a 2D image of a human face and a plane 
the human face, located forward, via operation key 204 
(e.g., cross key 209). Selection frame 701 may be dis- 
played in the form of a circle or a square. Furthermore, 
allowing the frame to be variable in size can help to des- 
ignate a subject accurately. Furthermore, ratherthan us- 
ing selection frame 701 surrounding a subject, using a 
pen (not shown) for inputto tracing and thus designating 
a subject to be cut out allows the subject to be cut out 
more accurately. 

[0047] Processor 1 05 having accepted the selection 
then uses edge detection or a similar image processing 
technique to cut the selected subject out of the 2D image 
and provides parallax for the cut-out subject's image to 
allow the subject to be seen more forward than a back- 
ground image other than the subject to create a 3D im- 
age. Although in Fig. 7Athe hair and the shoulders have 
a portion outside selection frame 701, processor 105 
preferably also estimates edge continuity for a portion 



15 



20 



25 



30 



35 



40 



45 



6 



11 



EP 1 379 063 A1 



12 



more or less outside the frame to cut out the portion. 
[0048] When in a vicinity of selection frame 701 the 
subject's image is cut out and a 3D image is created, as 
shown in Fig. 7B, the user perceives the subject's face 
in front of the plane. In the above described process, the 
three dimensional model described in the first embodi- 
ment is not used and the nose's height and other similar 
sense of depth of the face is not represented, whereas 
in the fourth embodiment processor 105 does not per- 
form a process related to a three dimensional model and 
it can readily create a 3D image. Note that in this case 
a subject is not limited to a human face and may for ex- 
ample be a building. 

[0049] Furthermore while in the above example after 
a picture has been taken selection frame 701 is used to 
select a subject to be seen forward. Alternatively, when 
a picture is to be taken selection frame 701 may be dis- 
played and the picture may be taken with a front subject 
positioned in a vicinity of selection frame 701 . This can 
eliminate the necessity of an operation designating se- 
lection frame 701 after a picture has been taken. Simply 
pressing a shutter key can display a 3D image. 
[0050] Furthermore while in the above example only 
a single subject is selected, a plurality of subjects may 
be selected, successively from a front subject. In that 
case, processor 105 provides each selected subject 
with different parallax information. Thus selecting a plu- 
rality of subjects and three dimensionally displaying the 
subjects can add further depth and hence realism. 
[0051] As has been described in each embodiment, 
camera 206 in the present embodiment for taking a pic- 
ture of a subject may be a camera used for picking up 
a typical 2D image and it is not required to have a spe- 
cific function for 3D image. As such it is not necessary 
to prepare a camera for picking up a 3D image and a 
camera used in a conventional mobile phone can exact- 
ly be used. More specifically in accordance with the 
present invention a 2D image picked up through a cam- 
era provided in mobile equipment can be displayed by 
the equipment three dimensionally. Since a single cam- 
era is sufficient, the mobile phone can be miniaturized 
and provide reduced power consumption. Furthermore, 
a significantly more degree of freedom in design for ar- 
ranging components can be provided than when two 
cameras for picking up 2D and 3D images are used. 
[0052] Furthermore in the above each embodiment 
the process up to creating a 3D image from a 2D image 
is entirely or partially performed by processor 1 05 read- 
ing software. Processor 105 may read a 3D conversion 
program previously stored in memory 106 and incorpo- 
rated in a mobile phone, or downloaded and thus ob- 
tained from a server. Furthermore the 3D conversion 
program may be used in a personal computer or other 
similar equipment other than mobile phones. 
[0053] Furthermore in the present mobile equipment 
the process described in each above embodiment may 
have its entirety or a portion combined with each other 
for execution. For example in the mobile equipment the 



process in the first embodiment for creating a 3D image 
from a three dimensional model and the process in the 
second embodiment for obtaining parallax information 
based on intensity in brightness of an image may be 
5 combined togetherfor execution. A more realistic3D im- 
age of a face together with a perspective background 
image can be displayed. 

[0054] Note that while the above mobile phone has 
been described as being foldable, it may be of a linear 
10 type. 

[0055] Furthermore while in the above description the 
mobile phone has been described as taking a still picture 
by way of example, it may be adapted to allow an ani- 
mated picture to be taken. This can be achieved simply 
15 by performing consecutively for a plurality of images 
forming the animated picture a process similar to that 
performed for a still picture. 
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Fifth Embodiment 



[0056] In converting into a 3D image a 2D still image 
for example of a user's face picked up through a digital 
camera, commercially available software such as the 
"PLACE 3 DPC" product kit manufactured by l-O DATA 

25 DEVICE, INC. described in the first embodiment allows 
a PC having mounted the software to take in the 2D still 
image and process the image. However, it is also pos- 
sible to convert the 2D image to a 3D image immediately 
by using software dedicated to 3D conversion that is 

30 provided in the body of the mobile phone in the fifth em- 
bodiment. In the fifth embodiment the mobile phone al- 
lows a 2D image to be converted to a 3D image, al- 
though pseudo in that an image for both eyes is not an 
actually picked-up image, and the 3D image to be three 

35 dimensionally displayed via commercially available soft- 
ware incorporated therein to provide conversion from a 
2D image to a 3D image, as described above. 



Sixth Embodiment 



[0057] While in the above embodiments a 2D image 
obtained by the mobile phone through a camera is con- 
verted to a 3D image, a 2D image (a photograph, ani- 
mation or the like) transmitted via a communication 

45 means can be received at communication control por- 
tion 1 01 and subjected to a process similar to that for a 
picked-up image, as described above, to provide a 3D 
image and display the image three dimensionally. This 
can be achieved simply by setting one operation key as 

50 a 3D image creation designating key. 

[0058] Thus the present mobile equipment can three 
dimensionally display a 2D image picked up through a 
camera provided in the mobile equipment. 
[0059] Furthermore in accordance with the present in- 

55 vention a camera equipped mobile phone can be used 
to take a picture of a subject and the picture can be dis- 
played three dimensionally on site immediately. A mo- 
bile phone is equipment always carried by the user and 
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without letting a chance to take a picture slip away the 
picture can be taken and the image can three dimen- 
sionally be displayed. A 3D display is more realistic than 
a 2D display and the obtained image can be more en- 
joyable. 

[0060] Although the present invention has been de- 
scribed and illustrated in detail, it is clearly understood 
that the same is by way of illustration and example only 
and is not to be taken by way of limitation, the spirit and 
scope of the present invention being limited only by the 
terms of the appended claims. 



Claims 

1 . Mobile equipment comprising: 

a pickup device (1 08) picking up an image of a 
subject; 

a three dimensional image creation portion 
(105) providing said image with parallax infor- 
mation to create a three dimensional image; 
and 

a display unit (1 09) displaying said three dimen- 
sional image. 

2. The mobile equipment of claim 1 , wherein said pick- 
up device (108) is a pickup device for picking up a 
two dimensional image. 

3. The mobile equipment of claim 2, further compris- 
ing: 

a communication control portion (101) control- 
ling a communication with another device; and 
a transmission and reception portion 
(102,103,104) allowing a call to and from an- 
other device. 

4. Mobile equipment comprising: 

a single pickup device (1 08) picking up an im- 
age of a subject; 

a three dimensional image creation portion 
(105) providing said image with parallax infor- 
mation to create a three dimensional image; 
and 

a display unit (1 09) displaying said three dimen- 
sional image. 

5. The mobile equipment of claim 4, wherein said pick- 
up device (108) is a pickup device for picking up a 
two dimensional image. 

6. The mobile equipment of claim 5, further compris- 
ing: 

a communication control portion (101) control- 



ling a communication with another device; and 
a transmission and reception portion 
(102,103,104) allowing a call to and from an- 
other device. 

7. The mobile equipment of claim 4, wherein said 
three dimensional image creation portion (1 05) cuts 
a human face out of said image picked up, to obtain 
a face image and provides said face image with said 
parallax information. 

8. The mobile equipment of claim 7, wherein said pick- 
up device (108) is a pickup device for picking up a 
two dimensional image. 

9. The mobile equipment of claim 8, further compris- 
ing: 

a communication control portion (101) control- 
ling a communication with another device; and 
a transmission and reception portion 
(102,103,104) allowing a call to and from an- 
other device. 



25 10. The mobile equipment of claim 4, wherein said 
three dimensional image creation portion (105) se- 
lects a particular subject from said image picked up, 
to obtain an image of the subject and provides said 
image of the subject and an image included in said 
image picked up other than said particular subject 
with different parallax information, respectively. 



11. The mobile equipment of claim 10, wherein said 
pickup device (108) is a pickup device for picking 
up a two dimensional image. 
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1 2. The mobile equipment of claim 1 1 , further compris- 
ing: 

a communication control portion (101) control- 
ling a communication with another device; and 
a transmission and reception portion 
(102,103,104) allowing a call to and from an- 
other device. 

13. A three dimensional conversion program product 
causing a computer to execute a three dimensional 
conversion process comprising the steps of: 

inputting a two dimensional image (S401); 
cutting a human face image out of said two di- 
mensional image to obtain a face image 
(S404); 

providing said face image with parallax infor- 
mation to create a three dimensional image 
(S406); and 

outputting said three dimensional image 
(S408). 
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FIG. 7 A 
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